Skip to content

Feature: validate typed scalar expression inputs (#57)#58

Merged
dannymeijer merged 6 commits into
mainfrom
feature/57-typed-scalar-validation
Jun 3, 2026
Merged

Feature: validate typed scalar expression inputs (#57)#58
dannymeijer merged 6 commits into
mainfrom
feature/57-typed-scalar-validation

Conversation

@dannymeijer

@dannymeijer dannymeijer commented Jun 2, 2026

Copy link
Copy Markdown
Member

Summary

This PR implements InQL issue #57 by making scalar helper contracts typed, schema-aware, and source-driven. Public helpers now declare precise ValueOrColumn inputs and typed expression results where InQL knows the contract; Prism carries row-schema facts forward; Substrait lowering rejects schema-incompatible column references before backend execution. It also updates the cast family to use Incan primitive type-token targets such as cast(col("amount_text"), float) and try_cast(col("status"), int) while keeping explicit string targets for compatibility spellings.

Type of change

  • Bug fix
  • New feature
  • Refactor / maintenance
  • Documentation
  • CI / tooling
  • RFC (adds/updates docs/rfcs/*)

Area(s)

Select the primary areas touched (labels sync from checked lines when the triage workflow runs):

  • Package & tests
  • Specification (RFCs)
  • Documentation
  • Automation & repo config
  • Other

Key details

  • User-facing behavior: Helpers now accept natural primitive values or typed column expressions at typed boundaries. For example, gt(col("qty"), 2), substring(col("sku"), 1, 3), make_date(2026, 5, 30), and cast(col("amount_text"), float) are the intended shapes. Broad lit(...) remains a structural escape hatch, not a way to bypass typed helper contracts.
  • Source contract cleanup: Removes registry-side scalar argument/result metadata and derives scalar contracts from checked public helper signatures instead. The registry still owns identity, lifecycle, policy, alias, aggregate/window/generator, and Substrait mapping metadata.
  • Typed expression model: Adds typed scalar wrappers such as boolean, integer, number, string, timestamp, and temporal column expressions. Helpers return concrete typed wrappers where their result is known.
  • Schema validation: Prism/Substrait lowering validates typed helper inputs against the current query schema before the backend adapter sees the plan. Backend inability remains backend-specific; type-contract violations are InQL planning diagnostics.
  • Cast DX: cast and try_cast now support primitive type-token overloads for bool, int, float, and str; safe_cast is an Incan alias of try_cast plus registry compatibility alias metadata. String target spellings remain for compatibility aliases such as int64 and float64.
  • Function coverage: Tightens contracts across operators, predicates, math, strings, regex, encoding, hashing, datetime, JSON/CSV/URL format helpers, nested data helpers, aggregate/window inputs, and examples/tests.
  • Incan dependency: This PR currently depends on dannys-code-corner/incan#751 / 0.3.0-rc47. CI is temporarily pinned to feature/750-primitive-type-tokens; after #751 lands on release/v0.3, this should be switched back to release/v0.3 with the same expected rc or final 0.3 version.
  • Review finding record: Source-quality findings were recorded under .agents/state/review-report.incan-source-quality.md locally and snapshotted in /tmp/incan-review-findings/2026-06-03-branch-feature-57-typed-scalar-validation-incan-source-quality-followup.md.

Testing / verification

  • make fmt INCAN=<rc47 compiler> -> 0 files changed
  • make fmt-check INCAN=<rc47 compiler> -> 276 src, 20 tests, 8 examples already formatted
  • make test-style -> passed
  • make registry-metadata INCAN=<rc47 compiler> -> 199 helpers
  • make build INCAN=<rc47 compiler> -> passed
  • make test INCAN=<rc47 compiler> -> 216 passed
  • make build-locked INCAN=<rc47 compiler> -> passed
  • make test-locked INCAN=<rc47 compiler> -> 216 passed
  • make smoke-consumer INCAN=<rc47 compiler> -> passed after allowing Cargo network access
  • git diff --check

Focused verification during the review/fix loop:

  • incan test tests/test_session_filters.incn -> 6 passed
  • incan test tests/test_session_projection.incn -> 17 passed
  • incan test tests/test_substrait_plan.incn -> 50 passed
  • incan test tests/test_function_registry.incn -> 19 passed
  • incan test tests/test_core_scalar_functions.incn -> 5 passed
  • incan test tests/test_common_scalar_functions.incn -> 5 passed
  • incan test tests/test_nested_data_functions.incn -> 3 passed

Docs impact

  • No docs changes needed
  • Docs updated
  • Docs follow Divio intent (tutorial/how-to/reference/explanation) where applicable

If docs updated:

  • Links: README.md, docs/language/reference/functions/index.md, builder/reference docs, nested function reference docs, and docs/release_notes/v0_1.md.

Checklist

  • I kept public docs user-focused and moved internals to contributing docs when appropriate
  • I avoided duplicating canonical install/run instructions in multiple places
  • I added/updated tests where it materially reduces regressions

Closes #57

@incan-triage-bot incan-triage-bot Bot added documentation Improvements or additions to documentation package Library source, tests, incan.toml labels Jun 2, 2026
@dannymeijer dannymeijer marked this pull request as ready for review June 2, 2026 19:03
@incan-triage-bot incan-triage-bot Bot added the automation CI, Makefile, .github/, repo config label Jun 2, 2026
@dannymeijer dannymeijer self-assigned this Jun 2, 2026
@dannymeijer dannymeijer added this to @InQL Jun 2, 2026
@dannymeijer dannymeijer added this to the InQL-v0.1 milestone Jun 2, 2026
@github-project-automation github-project-automation Bot moved this to Todo in @InQL Jun 2, 2026
@dannymeijer dannymeijer merged commit 9c2c98e into main Jun 3, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in @InQL Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation CI, Makefile, .github/, repo config documentation Improvements or additions to documentation package Library source, tests, incan.toml

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

feature - validate typed scalar expression inputs against query schema

1 participant